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I. Executive Summary and Overall Evaluation 

The 2015 Space Human Factors Engineering (SHFE) Standing Review Panel (from here on 
referred to as the SRP) met for a site visit in Houston, TX on December 2 - 3, 2015. The SRP 
reviewed the updated research plans for the Risk of Inadequate Design of Human and 
Automation/Robotic Integration (HARI Risk), the Risk of Inadequate Human-Computer 
Interaction (HCI Risk), and the Risk of Inadequate Mission, Process and Task Design (MPTask 
Risk). The SRP also received a status update on the Risk of Incompatible Vehicle/Habitat 
Design (Hab Risk) and the Risk of Performance Errors Due to Training Deficiencies (Train 
Risk). 

The SRP is pleased with the progress and responsiveness of the SHFE team. The presentations 
were much improved this year. The SRP is also pleased with the human-centered design 
approach. 

Below are some of the more extensive comments from the SRP. We have also made comments 
in each section concerning gaps/tasks in each. The comments below reflect more significant 
changes that impact more than just one particular section. 

Updates to the Human Research Roadmap Revision 

The SRP agrees that the Human Research Roadmap (HRR) needs to be updated. Most of the 
evidence reports, if not all, are inward-looking documents that rely heavily on documented and 
anecdotal experiences from past missions to identify risks and gaps, but that fail to look 
comprehensively at work systematically done elsewhere in related areas, whether in the military, 
academics or industry, including research-based findings (e.g., speech alarms), state-of-the-art 
technological solutions and consensus standards. Although the argument might be made that 
these efforts done elsewhere are not applicable to the space environment or the astronaut 
population (who are highly motivated, trained, etc.), there is no evidence to suggest these efforts 
be ignored because they cannot inform or be adapted to NASA's various missions. 

Arguably, this is a fundamental flaw in the entire risk and gap assessment process as the key 
decisions of what constitutes a risk, a gap and the appropriateness of proposed tasks to address 
are based on incomplete, potentially outdated evidence. 

As a result, the SRP would argue NASA should not only update their evidence reports but follow 


this by comprehensively reevaluating the risks, gaps and tasks based on the updated evidence. 
The current gaps are not equal; some are true research gaps; others are more technology 
acquisition gaps and some can be answered using literature reviews. This is fine but should be 
made more explicit. Additionally, literature review and technology acquisition tasks could 
possibly be funded using other mechanisms. It would be helpful to include the question being 
answered in the definition of each task which would support an easier understanding of the 
closure of the gaps. The SRP also encourages SHFE to explore opportunities to double up on 
tasks (e.g., can task models for design be reused as content to document user procedures and 
support training?). 

We encourage NASA to send SHFE scientists to more scientific conferences and meetings to 
keep current on the work being done. We have indicated in specific sections work that we kn ow 
of that is relevant to SHFE. 

Guidelines as deliverables from HARI/HCI tasks 

The SRP feels that guidelines are not the best deliverables as they have not proven to be 
effectively useful in these fields, in part because: 1) they do not support tradeoffs, and 2) they 
can be difficult to translate to verifiable implementations. Instead it would be more useful for 
the research to deliver models that support requirements development and comparison studies. 
Both HARI and HCI tasks should make use of these. 

Virtual Ground Controller 

The SRP thinks that a virtual ground controller needs to be designed and built. This is currently 
a planned task within HARI-02 but the SRP thinks that this task should address more than just 
enabling automation. This is not a research task but should capture the knowledge that the 
ground controllers have, as well as processes and tasks that are being designed for use in space, 
including the different automation processes and human-robot tasks. This necessitates designing 
a means for quick and accurate access into the knowledge base. Additionally, this software 
module should be designed to function as a training mechanism during travel. 

Automation and human-robot interaction 

The SRP feels it may be helpful to conceptually frame and define tasks addressing HARI gaps in 
terms of a hierarchy in which automation is considered a higher level system comprised of 
robotics, humans, human-robot teams or other equipment and which functions to either complete 
or help to complete steps in a task or process originally perfonned entirely by a person. Robotics 
is an important and significant subset of components in this hierarchy but commonly automated 
tasks such as data collection, storage, and analysis are often performed by other components such 
as sensors and computers. Another important subset of components are those particular tools 
that can perform one or more tasks with varying degrees of human intervention or collaboration, 
ranging from human operation of manual tools to no human involvement whatsoever (i.e., 
autonomous operations). The SRP notes that autonomous robots and autonomous systems are 
differentiated by several characteristics: the length of time of the operation and the transparency 
of the operations; robotic tasks are generally shorter and consist of physical actions that can be 
observed to determine if the task is being conducted correctly. This is discussed in more detail 
within the HARI Risk review below. 
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Also, the SRP would like to emphasize that any automated system, regardless of the type of 
components and collaborations utilized, should be resilient and adaptive. If a component (e.g., 
software, robot or human) fails or becomes impaired in some fashion, another component (e.g., 
human or robot) should be able to adapt and take on more of the task to ensure it gets 
accomplished. Many of the comments below with respect to the HARI Gap reflect this 
necessary design objective. 

Possible merging of the HARI and HCI Gaps 

The SRP sees much overlap in the HARI and HCI Gaps and tasks. The original HCI tasks 
looked more at the human side of the interface -e.g., vibration, visibility, audibility, motor 
control, and the control consequences of these interfaces (spatial orientation miscues, robot 
control errors, docking failures, etc.). Perhaps narrowing HCI to just this function will tighten 
the distinction (and make HCI a proper sub task of HARI). We realize that originally HARI was 
embedded within HCI but as the tasks have focused more on the interfaces and interactions the 
distinction between these two domains has become less clear. We encourage the SHFE Project 
to look into this prior to the next SRP review. 

If these domains are not merged, the overlaps should be addressed in the gaps/tasks and should 
definitely be noted. Additionally, meetings between these groups (perhaps including the TASK 
group) should be more frequent. 

Finally, the SRP would like to know who is responsible for software engineering and human 
systems integration by next year’s meeting. 

II. Critique of Gaps and Tasks for the Risk of Inadequate Design of Human 
and Automation/Robotic Integration (HARI Risk) 

A. Have the proper Gaps been identified to mitigate the Risk? 

a. Are all the Gaps relevant? 

b. Are any Gaps missing? 

B. Have the gap targets for closure been stated in such a way that they are measureable and closeable? 

a. Is the research strategy appropriate to close the Gaps? 

C. Have the proper Tasks been identified to fill the Gaps? 

a. Are the Tasks relevant? 

b. Are there any additional research areas or approaches that should be considered? 

c. If a Task is completed, please comment on whether the findings contribute to addressing or 
closing the Gap. 

D. If a Gap has been closed, does the rationale for Gap closure provide the appropriate evidence to support 
the closure? 

Gaps and Tasks: 

• The SRP thinks this Gap should discuss the different types of automation (Human/robot 
collaboration) and notes that the kind of autonomy likely required for long duration 
mission is a continuum, not discrete levels and is likely to be required to be adjustable. 
See general comment about automation. 

• Intelligent robots currently being researched in the artificial intelligence (AI) community 
have additional problems with human communication and interaction that are much more 
complex than for current forms of automation, and it is too early to be confident of 
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research outcomes. Whether this level of automation is or should be included in the 
HARI effort is a question that needs to be addressed and explicitly answered. For 
example, better conventional automation may solve many of the problems that would 
otherwise require human-like intelligence. 

HARI-01: We need to evaluate, develop, and validate methods and guidelines for 
identifying human-automation/robot task information needs, function allocation, and team 
composition for future long duration, long distance space missions. (Previously: What 
guidelines and tools can we develop to enable system designers and mission planners to 
conduct systematic task/needs analyses at the appropriate level of detail to allocate work 
among appropriate agents (human and automation)?) 

• The SRP thinks this Gap is appropriate and relevant. 

• The SRP recommends removing the word “guidelines” from the Gap title. 

Tasks: 

• Needs Assessment and Work Allocation Tools for Mission Operations and Procedures - 
Completed Task 

• Automation in Procedures: Guidelines for Allocating Tasks for Performance - PI: Debra 
Schreckenghost - TRACLabs, Inc. 

• Methods for Objective Robotic Functional Allocation for Exploration Missions - Planned 
Task 

• Space Human Factors and Habitability MIDAS-FAST: Development and Validation of a 
Tool to Support Function Allocation - Completed Task 

o There is almost no discussion of the current MIDAS-FAST technology. The SRP 
wonders if this tool can be adopted for wider HARI mission analysis 

• Spaceflight Robotic Operations, Allocations, and HARI Design Lessons Learned - 
Planned Task 

• Human- Automation-Robotics Allocation Verification Tool - Planned Task 

o The SRP thinks this task is central to the success of HARI design. Can the 
planned task be broadened to consider aspects that overlap HCI function 
allocation, too? There is substantial research on that and the SRP thinks both 
should begin with a literature review. Planning robotics missions will involve 
HCI. 

• Quantifying and Developing Countermeasures for the Effect of Latigue-Related Stressors 
on Automation Use and Trust During Robotic Supervisory Control - PI: Debra 
Schreckenghost - TRACLabs, Inc. 

HARI-02: We need to develop design guidelines for effective human-automation-robotic 
systems in operational environments that may include distributed, non-collocated adaptive 
mixed-agent teams with variable transmission latencies. (Previously: How can 
performance, efficiency, and safety guidelines be developed for effective information 
sharing between humans and automation, such that appropriate trust and situation 
awareness is maintained?) 

• The SRP thinks this Gap is appropriate and relevant. 

• The SRP thinks that the designs of autonomous operations for long-term spaceflight are 
missing from this Gap. 
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• The SRP recommends rethinking the framework of how robotics and automation should 
be put together and thought about together. See the Executive Summary/Overall 
Evaluation (Section I above). 

• There is no unique requirement for tele-automation. If it works on Earth, will it 
automatically work on Mars? 

• The SRP suggests an integration/discussion between the NASA Jet Propulsion 
Laboratory (JPL) (robotics expertise), HRP SHFE (human aspect) and NASA Langley 
Research Center (LaRC) (drone expertise). 

• The SRP recommends adding a task dealing with safety. A literature review in this area 
should be carried out and should include work in both industrial and non-industrial 
robotic applications as well as the relevant standards (e.g., American National Standards 
Institute/Robotics Industries Association (ANSI/RIA) R15.06 - the Industrial Robot 
Safety Standard, which adopts the International Organization for Standardization (ISO) 
10218 robot safety standards). Of particular interest may be the current standardization 
efforts for safety of collaborative robots: see the ISO/TS 15066 technical specification 
and the ongoing efforts from the various Joint Working Groups between the International 
Electrotechnical Commission (IEC) and ISO (under the ISO Technical Committee (TC) 
184/Subcommittee (SC) 2). 

Tasks: 

• Assessing and Mitigating the Impact of Transmission Delays on Teleoperations - 
Completed Task 

• Assessment, evaluation, and development of methodologies, metrics, and tools available 
for use in multi-agent (human and robotic) teaming - Completed Task 

• Design tools for automation architectures in support of distributed control teams - 
Planned Task 

• Multimodal Augmented Displays for Surface Telerobotic Missions - PI: Elizabeth 
Wenzel, Ph.D. - NASA Ames Research Center 

o It was not clear to the SRP that this task is meaningful or new (as research). 
Perhaps it just required better motivation for the specific NASA mission 
applications anticipated. 

• Information Visualization and Controls for HRI - Planned Task 

• Quantification and Mitigations for Loss of Situation Awareness during Transitions 
between Levels of Automation - Planned Task 

o The SRP recommends doing a literature search for this. 

o The SRP also suggests changing this task name to match what is being done. 

• Advanced Displays for Efficient Training and Operation of Robotic Systems (Robotic 
Systems-Oman, Completed) - Completed Task 

• Task Difficulty Measure for the Design of Telerobotics Space Operations - Planned Task 

• Command and Control Design for Spacebound Flyers - Planned Task 

• Command and Control Design for Highly Dexterous Robots - Planned Task 

• Technology and State of the Art of Spacecraft Automation: Command, Control, and Fault 
Detection, Isolation, & Resolution - Planned Task 

• Enabling Automation for Crew Autonomy: Virtual Ground Controller - Planned Task 
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HARI-03: We do not know how to quantify overall human-automation-robotic system 
performance to inform and evaluate system designs to ensure safe and efficient space 
mission operations. (Previously: How can performance, efficiency and safety guidelines be 
developed for appropriate task automation and the effective allocation of tasks between 
humans and automation?) 

• The SRP thinks this Gap is relevant and appropriate. 

• The SRP thinks much of this work can be done as “library work”. 

• The SRP notes that robotic interaction research funded by the U.S. Army and the Office 
of Naval Research (ONR) is highly relevant and should be explored in addition to current 
interaction with Air Force Research Laboratory (AFRL). 


Tasks: 

• Automation Trust and Complacency - Planned Task 

o The SRP thinks this task is very important and needs to be accomplished as soon 
as possible. 

o This task impacts how the human/robot tasks get allocated, 
o Research in this area is also being funded by the U.S. Army and ONR. 

• Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human- 
Automation Systems - PI: Eduardo Salas, Ph.D., University of Central Florida 

• Metrics and Methods for Real-Time Task Performance Assessment - PI: Kevin Duda, 
Ph.D., The Charles Stark Draper Laboratory, Inc. 

• Mixed-Agents System Performance Methods and Measures - Planned Task 

• Verification Tools for Successful Human- Automation Integration in Operational Space 
Systems - Planned Task 

• Verification Tools for Successful Human-Robotic Integration in Operational Space 
Systems - Planned Task 

• Automation Interface Design Tools Development - Completed Task 

• Human - Automation Interactions and Performance Analysis of Lunar Lander 
Supervisory Control - Completed Task 

• Effective On-Board Training for Dynamic Operations - Planned Task 

SHFE-HARI-04: What are the effects of the delays typical of different mission regimes on 
teleoperations and how do we mitigate these effects? (MERGED with SHFE-HARI-02) 

• The SRP thinks this Gap is relevant and appropriate. 

Task: 

• Assessing and Mitigating the Impact of Transmission Delays on Teleoperations - 
Completed Task 

HARI-04: We need to identify and scope the critical human-automation/robotic mission 
activities and tasks that are required for future long duration, long distance space missions, 
(formerly SHFE-HARI-05) 

• The SRP thinks this Gap is relevant and appropriate. 

Tasks: 
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• Human-automation and Robotic Integration Design Reference Mission Task Scoping - 
Planned Task 

• Human-automation and Robotic Integration Trade Study Analysis - Planned Task 

III. Critique of Gaps and Tasks for the Risk of Inadequate Human-Computer 
Interaction (HCI Risk) 

A. Have the proper Gaps been identified to mitigate the Risk? 

a. Are all the Gaps relevant? 

b. Are any Gaps missing? 

B. Have the gap targets for closure been stated in such a way that they are measureable and closeable? 

a. Is the research strategy’ appropriate to close the Gaps? 

C. Have the proper Tasks been identified to fill the Gaps? 

a. Are the Tasks relevant? 

b. Are there any additional research areas or approaches that should be considered? 

c. If a Task is completed, please comment on whether the findings contribute to addressing or 
closing the Gap. 

D. If a Gap has been closed, does the rationale for Gap closure provide the appropriate evidence to support 
the closure? 


Gaps and Tasks: 

• The SRP thinks the Risk may need to be broader to include more about the importance of 
teamwork. The lack of interoperability among systems may be an HCI Risk because it 
creates usability problems for teamwork and error opportunity by forcing crew to perform 
overhead tasks for summary/transcription of information sharing. The SRP recommends 
(if possible) NASA put someone on the Standards Coordinating Council for 
interoperability. 

• The SRP thinks the majority of the Gaps are relevant and appropriate to mitigate the 
Risk. 

• There are some Gaps the SRP thinks are missing and some that lack focus. There 
appears to be too much reliance on guidelines as deliverables rather than on models that 
support tradeoffs of different design options and are detailed enough to specify what 
needs to be implemented in the system functions and user interface. 

• Standards like Object Management Group’s (OMG) Business Process Modeling Notation 
define model contents in terms of data types. The scope of an HCI design can be defined 
in terms of the Conceptual Work Product it must create and the context where the work 
will be performed. 

• The SRP thinks the information the models need to contain needs to be identified based 
on the question the model is trying to answer and then identify how it will be used. 

• The HCI work seems heavy on the display side and not as focused on controls. The SRP 
thinks controls will be an issue - fine motor control, vibration impact. 

• The SRP thinks user centered design (UCD) must be an integral part of the systems 
design effort. In order to achieve human-systems integration, the users’ procedures must 
be designed as carefully as the machine’s because they must all work together in a 
complementary way to produce the needed conceptual work product. Concurrent 
engineering methods for this are described in the literature. Concurrent engineering 
techniques address this by using design representations that allow each of the two (or 
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more) perspectives to understand how a design decision in one will affect the other, then 
proceed iteratively to converge on a single design that "satisfies" both. 

• Working with MPTASK-02 is a good place to start. The bulk of the studies are task- 
specific. The SRP thinks that given the lack of current knowledge about the 
missions/processes/tasks to be supported that the focus should be more on a user-oriented 
“front end analysis”. 

• Missing Gap: as with HARI, function allocation is a major design decision that should 
precede user interface design. Generally, the HCI Gaps focus too narrowly on the user 
interface. Information architecture and function allocation must also have major impact 
crew task/process performance requirements. If they don’t satisfy crew performance 
needs they can’t be fixed in user interfaces. 

• HCI-05 Gap is too narrow: HCI evaluation also needs verification of effectiveness and 
safety of large complex systems, and from a practical standpoint to verify equivalence of 
design options to enable tradeoff analyses. 

• Overall, it is difficult to see how the tasks for HCI-03 add up to address the Gap. For all 
HCI gaps it will really help relate tasks to gaps if the task descriptions state the question 
it is trying to answer. The questions may also reveal tasks that can do double or even 
triple-duty, e.g., for HARI, MPTASK or training. 

• Overall, the organization of HCI Gaps and tasks would benefit by mapping them to the 
structure of ISO 9241-210:2010 (superseded ISO 13407). 

HCI-01: We need to understand the effects of vibration and acceleration on crew task 
performance, and how those effects can be mitigated? (Previously SHFE-HCI-01 - What 
are the effects of vibration and acceleration on crew task performance and how can those 
effects be mitigated?) 

• The SRP thinks this Gap is relevant and appropriate. 

Tasks: 

• Effects of Vibration on Performance in a Spacesuit - Planned Task 

• Analysis of Vibration data from Orion Exploration Mission 1 (EMI) - Planned Task 

• Robust Human-System Interface Design for Spaceflight-Induced Environments (Interface 
Design-Stone, Completed) - Completed Task 

HCI-02: We need to understand what aspects of cognitive function and fine motor skills 
change during long-duration missions and how these changes affect task performance, (was 
SHFE-HCI-02) 

• The SRP thinks this Gap is relevant, but it is not clear what the cognitive aspect of the 
Gap is. 

• The Behavioral Health and Perfonnance (BHP) Element is looking at autonomy in long- 
duration flights and different types of workload measures. 

Tasks: 

• Effects of Long-duration Microgravity on Fine Motor Control Skills - PI: Kritina 
Holden, Ph.D., NASA Johnson Space Center 

• Effects of Attentional Variations and Cognitive Load on Long-duration Task 
Perfonnance - Planned Task 
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Cognitive Scoping - Planned Task 
Interactive Cognitive Aids - Planned Task 






HCI-03: We need HCI guidelines (e.g., display configuration, screen-navigation) to mitigate 
the performance decrements and operational conditions of long duration spaceflight. 
(Formerly SHFE-HCI-03) 

• The SRP does not think this is a Gap. This Gap covers traditional human factors for 
spaceflight. 

• The SRP recommends changing the gap title to: “We need HCI requirements guidelines 
(e.g., display configuration, screen-navigation) to mitigate the performance decrements 
and operational conditions of long duration spaceflight. ” 

• The task for Interactive Cognitive Aids is duplicated in HCI-02. 

• Several of the tasks, such as Bedrest and Spaceflight Effects appear to need close 
coordination with the BHP Element. 

Tasks: 

• Advanced multimodal solutions for information presentation - Planned Task 

• Analog Validation of Multimodal Information Systems - Planned Task 

• Speech Alarms Pilot Study - PI: Aniko Sandor, Ph.D., NASA Johnson Space Center 

• Multimodal Augmented Displays for Surface Telerobotic Missions - PI: Elizabeth 
Wenzel, Ph.D., NASA Ames Research Center 

• Displays and Controls Interfaces - Completed Task 

• Enhancement of Spatial Orientation Capability of Astronauts on the Lunar Surface - 
Completed Task 

• Information Presentation - Controls Technology Survey and Testing - Completed Task 

• Infonnation Presentation - Displays Development (Visual and Auditory) - Completed 
Task 

• Information Presentation - Electronic Procedures and Fault Management - Completed 
Task 

• Infonnation Presentation - Human Performance Modeling - Completed Task 

• Modeling and mitigating spatial disorientation in low g environments - Completed Task 

• Sensorimotor Displays and Controls to Enhance the Safety of Human/Machine - 
Completed Task 

• Developing Predictive Measures of Sensorimotor Adaptability to Produce Customized 
Countermeasure Prescriptions (Bloomberg) - PI: Jacob Bloomberg, Ph.D., NASA 
Johnson Space Center 

• Bed Rest as a Spaceflight Analog to Study Neurocognitive Changes: Extent, Longevity, 
and Neural Bases (Seidler) - PI: Rachael Seidler, Ph.D., University of Michigan 

• Spaceflight Effects on Neurocognitive Performance: Extent, Longevity, and Neural Bases 
(NeuroMapping-Flight-Seidler, Active) - PI: Rachael Seidler, Ph.D., University of 
Michigan 

• The Relative Benefits of the Auditory, Visual and Tactile Channels for Obstacle 
Avoidance During Surface Exploration Missions - Planned Task 

• Task Analysis Visualization Tools - Planned Task 

• Procedure Selection for Tasks - Planned Task 
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• Evaluation of Crew-Centric Onboard Mission Operations Planning and Execution Tool - 
PI: Steven Hillenius, Ph.D., NASA Ames Research Center 

• Communication tools for scheduling - Planned Task 

• Interactive Cognitive Aids - Planned Task 

HCI-04: We need to understand how emerging multi-modal and adaptive display and 
control technologies are best applied to the design of HCI for proposed long-duration DRM 
(Design Reference Missions) operations. (Was SHFE-HCI-04) 

• The SRP thinks this is a technology acquisition Gap. 

• This Gap also means tracking and anticipating where new user interface (UI) technology 
will be by the date of the infrastructure freeze so they know how to design effectively 
with it (e.g., sensor suites of intemet-of-things may replace input technologies of 
conventional user interfaces). 

• The SRP does not think the title of the Gap matches what is being done. 

• The SRP does not think multi-modal/speech alarms are the point of the Gap, but rather 
how HCI design can be adapted for long duration spaceflight. 

Tasks: 

• Tech Watch for Improving HCI in Next-Generation Missions- 1 - Planned Task 

• Tech Watch for Improving HCI in Next-Generation Missions-2 - Planned Task 

• Tech Watch for Improving HCI in Next-Generation Missions-3 - Planned Task 

• Tech Watch for Improving HCI in Next-Generation Missions-4 - Planned Task 

• Tech Watch for Improving HCI in Next-Generation Missions-5 - Planned Task 

HCI-05: We need verifiable requirements that specify standard measurement techniques 
and metrics for evaluating the quality of user interfaces with specific attention to the 
usability and evolvability of an interface. (Was SHFE-HCI-05) 

• The SRP thinks the term “verifiable requirements” should be defined as “the 
requirements should be stated in a way that allows the design to be evaluated against 
them to determine if the design satisfies the requirements.” The Gap Rationale says 
“long-duration”, but the SRP thinks this is important for any mission. 

• The SRP thinks there is significant literature on this topic. 

• The SRP recommends using the common industry standard ISO 25062 for usability 
testing. 

• The SRP thinks ISO 25062 should become part of NASA’s solicitation process to ensure 
that equipment and systems can be operated successfully by qualified crew. 

• The SRP thinks the “Targets for Closure” for this Gap (“verifiable requirement”) are 
appropriate. 

• The SRP strongly recommends replacing guidelines with models that support tradeoff 
studies and can be translated more directly into implementation. 

• The SRP thinks this Gap needs to be expanded to include the state-of-the-art in modeling 
efforts and determine what is specific in a Mars mission that isn’t supported. 
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Tasks: 


• Human Factors Analysis Support Tool (H-FAST) Phase III - Completed Task 

• Human Factors Analysis Support Tool (H-FAST) - Completed Task 

• Usability Evaluation - Completed Task 

• Human-Centered Design in an Agile Process - Planned Task 

HCI-06: We need guidelines to ensure crewmembers receive all of the information required 

to accomplish necessary tasks in a timely fashion, even when operating autonomously. 

(Was SHFE-HCI-06) 

• The SRP thinks this Gap is relevant and appropriate. 

• The SRP encourages consistent interfaces for guidelines. 

• The SRP thinks this Gap also needs to address having the necessary situation awareness 
during semi- autonomy/autonomy. 

• This Gap is closely related to the HARI and Train Risks. 

• The SRP thinks there is a task missing related to not knowing how to design HCI for 
effective teamwork. There may be a need to make teamwork its own Gap or at the very 
least, collaborate with the BHP Element Team group to address this. 

Tasks: 

• Long-duration Information Systems - Planned Task 

• Electronic Procedures for Crewed Missions Beyond Low Earth Orbit (LEO) - PI: 

Kritina Holden, Ph.D., NASA Johnson Space Center 

• Long-duration Information Systems - Validation - Planned Task 

• Validation of Autonomous information system - flight - Planned Task 

• Electronic Procedures for Autonomous Crews - Analog Validation - Planned Task 

• Electronic Procedures Validation - Orion - Planned Task 

o The SRP was told by the SHFE Project that the Astronaut Office has a big impact 
on the Orion display development team. The SRP agrees that the astronauts 
should definitely be involved in the design process but notes that their input 
should be weighted and integrated with the best practices in human 
factors/ergonomics. While past experience is an integral part of the design, it 
should not be the sole determinate of an HCI/HARI system facilitating future, 
u nkn own applications. This is an example of the use of where tradeoff analysis 
should be used to determine design decisions. 

• Advanced Procedures Consortium - Planned Task 

• Task Analysis Visualization Tools - Planned Task 

• Procedure Selection for Tasks - Planned Task 

• Design and Evaluation of Automated Electronic Checklists for Robotics Operations - PI: 
Charles Oman, Ph.D., Massachusetts Institute of Technology. 

o The SRP thinks there should be a NASA Standard for electronic checklists that is 
developed with HARI, HCI and MPTASK. The gap seems to be the integration 
with knowledge sources, both data and procedures. The medical domain could be 
the focus problem, as it requires a complex mix of training (skills and skill 
development), knowledge (terms, symptoms, consequences), decision-making 
(opportunistic treatments based on resources), and evaluating/handling multiple 
possible outcomes. 
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o Tech Watch here should include the burgeoning area of online courses and 
whether or how these can be mapped to or adapted for spaceflight use. 

SHFE-HCI-07 (SM11): Can crewmember spatiomotor abilities be more accurately 
predicted and countermeasures and training techniques developed to mitigate spatial 
disorientation during spaceflight? (Merged with HCI-03) 

Tasks: 

• Enhancement of Spatial Orientation Capability of Astronauts on the Lunar Surface - 
Completed Task 

• Modeling and mitigating spatial disorientation in low g environments - Completed Task 

HCI-07: We need to define the acceptable level of risk for HCI performance relative to 
terrestrial baselines. (Was SHFE-HCI-08) 

• Although a relevant Gap, the SRP thinks this Gap should be combined with Gap HCI-02 
- difference between terrestrial and long duration missions. 

Tasks: 

• A Research Roadmap Approach to Defining the Risk of Inadequate HCI - Planned Task 

• Operational Knowledge/Evidence - Phase 1 - Planned Task 

• Operational Knowledge/Evidence - Phase 2 - Planned Task 

• Operational Knowledge/Evidence - Phase 3 - Planned Task 

IV. Critique of Gaps and Tasks for the Risk of Inadequate Mission, Process 
and Task Design (MPTask Risk) 

A. Have the proper Gaps been identified to mitigate the Risk? 

a. Are all the Gaps relevant? 

b. Are any Gaps missing? 

B. Have the gap targets for closure been stated in such a way that they are measureable and closeable? 

a. Is the research strategy’ appropriate to close the Gaps? 

C. Have the proper Tasks been identified to fill the Gaps? 

a. Are the Tasks relevant? 

b. Are there any additional research areas or approaches that should be considered? 

c. If a Task is completed, please comment on whether the findings contribute to addressing or 
closing the Gap. 

D. If a Gap has been closed, does the rationale for Gap closure provide the appropriate evidence to support 
the closure? 


Gaps and Tasks: 

• The SRP thinks that these Gaps have significant overlap. Moreover, there are numerous 
well-established and basic industrial engineering methods and tools that may be suitable 
for measuring performance at all of these levels, especially for MPTask-01 and MPTask- 
02. The SRP suggests starting with the work measurement literature for task and process 
level perfonnance; the engineering management literature could provide insight into how 
to assess mission-level performance. 
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• The SRP also would encourage more specificity when discussing cognitive tasks and 
cognitive load. Are they referring to a classification scheme in which the 
characteristics/parameters of “cognitive” tasks are described as opposed to “perceptual” 
or “motor” tasks? The SRP is also unsure of their working definition of “cognitive load.” 
Again, a literature search is needed in these areas. 

• The SRP suggests exploring the area of augmented cognition with respect to monitoring 
task or process perfonnance in a human-machine system but cautions that this is an area 
in which the validation research has not kept up with the technological advances. For 
example, while there has been progress made in the physiological monitoring of 
workload, little effort has been directed toward understanding how and when this should 
drive dynamic changes in the interface/function allocation or how these changes may 
affect the human operator. 

MPTASK-01: We need methods and tools to collect measures of mission, process, and task 
performance. (Previously SHFE-TASK-01) 

• The SRP thinks this Gap is relevant and appropriate but not as a research gap. 

• The SRP believes the efforts to develop methods of unobtrusive data collection is both 
good and appropriate. 

• The SRP would argue that the capability to objectively measure (1) the level of effort, (2) 
the quality of the effort, and (3) the inherent difficulty of the mission/process/task needs 
to be included in the research efforts. Ideally, these measures should be consistent and 
scalable (i.e., usable at any level: mission, process, or task). In addition, any 
measurement methods and tools developed also need to be evaluated for effectiveness, 
validity and reliability. 

• AFRL is doing a lot of this work. NASA should attempt to piggyback on what they are 
doing so as not to duplicate effort. 

• NASA LaRC is also doing similar work. The SRP suggests contacting them and having 
them do the work in their simulators (Randy Bailey). 

• The SRP has questions as to how cognitive tasks and cognitive loads are defined - see the 
Executive Summary/Overall Evaluation (Section I above). 

• The SRP observed that NASA seems to define workload differently than 
academia/industry; this is fine as long as their definition is clearly stated. 

• The development of task and process measures needs to be coordinated closely with 
measures for HCI since they often use task performance to evaluate a system or user 
interface. 

Tasks: 

• Integration of Workload Metrics as Constraints into NASA Scheduling Tools - Planned 
Task 

• Spaceprint: Development and Validation of a Tool to Predict, Evaluate, and Mitigate 
Excessive Workload Effects - Completed Task 

• Workload Tools and Guidelines - Completed Task 

• Evaluation of Crew-Centric Onboard Mission Operations Planning and Execution Tool - 
PI: Steven Hillenius, Ph.D., NASA Ames Research Center 
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• Integrate Workload Measures Questionnaire - Planned Task 

• Workload, Lighting, Scheduling, and C02 in Long Duration Mission Analogs - Planned 
Task 

• Workload and Scheduling Tools for Long Duration Missions (Research Requirements) - 
PI: Peter Hancock, Ph.D., University of Central Florida 

• Workload and Scheduling Tools for Long Duration Missions - Planned Task 

• Operational Knowledge/Evidence - Phase 1 - Planned Task 

• Operational Knowledge/Evidence - Phase 2 - Planned Task 

• Operational Knowledge/Evidence - Phase 3 - Planned Task 

• In-situ, unobtrusive methods for measuring crew resource management (CRM) and 
situation awareness (SA) - Planned Task 

• In-situ, unobtrusive methods for measuring workload - Planned Task 

• Validation of unobtrusive Crew Resource Management (CRM), Situation Awareness 
(SA), and Workload - Planned Task 

• Unobtrusive Workload - SBIR2 - Planned Task 

• Long-duration Information Systems - Planned Task 

• Long-duration Information Systems - Validation - Planned Task 

MPTASK-02: We need methods and tools to support mission, process, and task design. 

(Previously SHFE-TASK-02) 

• Again, the SRP does not feel that this is a research gap. 

• The SRP thinks it is difficult to separate the design of the mission, process and task 
design from the tool being used. Because tools constrain the way they can be used to 
perform tasks, the tasks and tools to perfonn them must be engineered concurrently. 

• This Gap is definitely related to autonomy. 

• The SRP recommends looking at different tools that are applicable - need to understand 
the resources and the constraints. 

• For this Gap, the SRP thinks that Business Process Model and Notation (BPMN) is a 
standard that can be used in the design of teamwork processes. Also, existing tools such 
as Kieras’ GLEAN and Bonnie John’s CogTool can be used to represent and evaluate 
task designs, and by incorporating other tools used in industrial engineering, the analysis 
can cover non-computer aspects of tasks. Also many of the comments on autonomy are 
appropriate when discussing dynamic allocation of tasks. 

Tasks: 

• Human Performance Data Project - PI: Jurine Adolf, Ph.D. - NASA Johnson Space 
Center 

• Generalizable Skills and Knowledge for Exploration Missions - PI: Jack Stuster, Ph.D. - 
Anacapa Sciences, Inc. 

o The SRP recommends adding a task success requirement to this project. 

• Integrating Externally Developed Models into HRP's "Virtual Astronaut" - Planned Task 

• Advanced Procedures for Autonomous Missions - Planned Task 

• Inventory Management - Planned Task 

• Task Analysis Visualization Tools - Planned Task 

• Procedure Selection for Tasks - Planned Task 
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• Interactive Cognitive Aids - Planned Task 

o The SRP thinks this task seems totally different than other planned/ current tasks, 
o The SRP thinks this could be part of the Train Risk. 

• Electronic Procedures for Crewed Missions Beyond Low Earth Orbit (LEO) - PI: 

Kritina Holden, Ph.D., NASA Johnson Space Center 

• Electronic Procedures Validation - Orion - Planned Task 

• Electronic Procedures for Autonomous Crews - Analog Validation - Planned Task 

• Advanced Procedures Consortium - Planned Task 

MPTASK-03: We need methods and tools for planning and dynamic re-planning of crew 
schedules. (Previously TASK-03) 

• The SRP thinks there is significant confusion between this Gap and Gap MPTask-02. 
These Gaps are redundant and should be combined. 

• The SRP recommends looking at reusable procedures for the design of task - should be 
considered in electronic procedures/training as well. 

• Once again, the SRP thinks this is closely related to the issues of function allocation and 
autonomy. 

Tasks: 

• Crew Scheduling Tools - Completed Task 

• Evaluation of Crew-Centric Onboard Mission Operations Planning and Execution Tool - 
PI: Steven Hillenius, Ph.D., NASA Ames Research Center 

• Workload, Lighting, Scheduling, and C02 in Long Duration Mission Analogs - Planned 
Task 

• Models to Inform Scheduling Decisions and Implementing Fatigue-Related 
Countermeasures - Planned Task 

• Workload and Scheduling Tools for Long Duration Missions - Planned Task 

• Workload and Scheduling Tools for Long Duration Missions (Research Requirements) - 
PI: Peter Hancock, Ph.D., University of Central Florida 

• Communication tools for scheduling - Planned Task 

MPTASK-04: We need to identify causal factors and sources of inefficiency related to 
inventory management and stowage for long-duration missions, as well as processes and 
tools to mitigate identified issues. 

• The SRP thinks this Gap is relevant and appropriate. 

• The SRP thinks this Gap warrants a literature review and is better described as a tool 
acquisition gap than a research gap. This is a specific problem, which most industrial 
engineers would address using known tools and methods. In addition, other than the 
extreme mission duration, this does not seem to be a unique problem - for example, the 
SRP suspects submariners from any nation might be able to lend further insight into these 
issues. 

• The SRP argues you need to know what tasks are being perfonned before you can deal 
with inventory management and stowage. 

• Ultimately, the SRP views this as a facility layout problem well-suited for a Small 
Business Innovation Research (SBIR). 
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Tasks: 


• Inventory Management - Planned Task 

• Inventory Management for Long Duration Missions (LDM) - Planned Task 

V. Discussion on the strengths and weaknesses of the IRP and identify 
remedies for the weaknesses, including answering these questions: 

A. Is the Risks addressed in a comprehensive manner? 

• The SRP thinks the Risks are addressed in a comprehensive manner and that both 
the SHFE discipline is doing a good job at trying to alleviate the Risks. 

VI. Evaluation of the progress on the HARI, HCI, and MPTask Risks 
Research Plans since the 2014 SRP meeting 

• The SRP is very impressed with the progress made in the IRP since the 2014 SRP 
meeting. 

VII. Additional Comments regarding the Risk of Incompatible 
Vehicle/Habitat Design (Hab Risk) Status Review 

• The SRP thinks this Risk has changed for the better. 

• The SRP is pleased with the interactions with the BHP Element. 

• NASA should continue to develop interconnected simulation tools; these 
would eventually integrate different BHP parameters, work schedules, 
environmental variables, habitable volume, and psychosocial situations. 

• Space is a huge problem and a case needs to be made for more funding in this 
area. With respect to the Net Habitable Volume (NHV), current efforts seem to 
focus on modeling current day tasks. But design decisions need to be based on 
models that utilize a suite of tasks yet to be defined. The SRP suggests that 
studies should look at multi-purpose space and the cost of reconfiguring space 
should it be necessary. 

• The Hab Risk needs to collaborate closely with MPTask-04 on stowage as this 
will have a big impact on NHV. 

• The SRP understands that the NHV for some fundamental tasks has already been 
computed. The SRP would like to know for which tasks this has been completed 
and, whether or not simultaneous tasks have been accounted for. The SRP 
assumes that NASA has identified some tasks where habitable volume concerns 
are crucial (e.g., medical procedures, exercise); are there others? We also assume 
that NASA experience may cover LEO tasks, so what new ones are anticipated 
for Mars or asteroid missions? Equipment repairs? 3D printing? Inventory 
access? Specifics would be helpful to understand and scope the problem 
parameters. In addition, anthropometries variations impact maneuvers in a 
volume. Do the NHV models account for this? 

• The NHV computational models currently under development appear to 
encompass a large number of parameters, which from a modeling standpoint can 
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complicate validation. Given the importance of these models for the various 
design efforts (e.g., spacecraft, workspace layout, task), is there a back-up plan if 
these models do not work out? Are there criteria in place to define whether or not 
an NHV model is “successful” (i.e., valid)? 

VIII. Additional Comments regarding the Risk of Performance Errors Due to 
Training Deficiencies (Train Risk) Status Review 

• Only three tasks are currently funded at this point. The SRP thinks more tasks 
need to get funded sooner rather than later. 

• The SRP thinks the SHFE Project should consider integrating the task design with 
the training as well as with the electronic procedures. This allows the training to 
leverage from both HCI/Design and electronic procedures. Also, this could be a 
form of evaluation for design procedures. 

• Consider supporting embedded training, in which the on-board systems include a 
simulation mode that can be used en route to train in several areas, such as: 

1 . Fault simulation to train in-depth problem solving in different systems. 

2. Continuous drills in handling non-normal situations. 

3. Support for crews developing their own training en route to enable cross- 
training and specialized training; this could include situations where one 
person trains another person and provides feedback on how they do the task. 

• The SRP would like to know how the SFIFE Project will train for unexpected 
situations. Could this be used as a crew selection procedure? The SHFE Project 
needs to overlap with the BHP Element on this possibility. 

• Training should consider the additional use of gamification - this would also give 
astronauts a fun way to train during the trip to Mars. 

• The SHFE Project needs to determine which types of tasks just-in-time 
information would suffice versus those where just-in-time training is needed. 
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IX. 2015 Space Human Factors Engineering SRP Research Plan Review: 
Statement of Task for the Risk of Inadequate Human-Computer 
Interaction, the Risk of Inadequate Design of Human and 
Automation/Robotic Integration, and the Risk of Inadequate Mission, 
Process and Task Design 


The 2015 Space Human Factors Engineering (SHFE) Standing Review Panel (SRP) is chartered 
by the Human Research Program (HRP) Chief Scientist. The purpose of the SRP is to review the 
Risk of Inadequate Human-Computer Interaction (HCI), the Risk of Inadequate Design of 
Human and Automation/Robotic Integration (HARI), and the Risk of Inadequate Mission, 
Process and Task Design (MPTASK) sections of the current version of the HRP’s Integrated 
Research Plan (IRP) which is located on the Human Research Roadmap (HRR) website 
(http://humanresearchroadmap.nasa.gov/) . Your report, addressing each of the questions in the 
charge below and any addendum questions, will be provided to the HRP Chief Scientist and will 
also be made available on the HRR website. 

The 2015 SHFE SRP is charged (to the fullest extent practicable) to: 

1 . Based on the information provided in the current version of the HRP’s IRP, evaluate the 
ability of the IRP to satisfactorily make progress in mitigating the Risk by answering the 
following questions: 

A. Have the proper Gaps been identified to mitigate the Risk? 

i) Are all the Gaps relevant? 

ii) Are any Gaps missing? 

B. Have the gap targets for closure been stated in such a way that they are measureable 
and closeable? 

i) Is the research strategy appropriate to close the Gaps? 

C. Have the proper Tasks been identified to fill the Gaps? 

i) Are the Tasks relevant? 

ii) Are there any additional research areas or approaches that should be considered? 

iii) If a Task is completed, please comment on whether the findings contribute to 
addressing or closing the Gap. 

D. If a Gap has been closed, does the rationale for Gap closure provide the appropriate 
evidence to support the closure? 

2. Identify the strengths and weaknesses of the IRP, and identify remedies for the weaknesses, 
including, but not limited to, answering these questions: 

A. Is the Risk addressed in a comprehensive manner? 

B. Are there areas of integration across HRP disciplines that are not addressed that would 
better address the Risk? 

C. Other 
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3. Based on the updates provided by the Element, please evaluate the progress in the research 
plan since the last SRP meeting. 

4. Please comment on any important issues that are not covered in #1, #2, or #3 above, that the 
SRP would like to bring to the attention of the HRP Chief Scientist and/or the Element. 

Additional Information Regarding This Review: 

1 . Expect to receive review materials at least four weeks prior to the meeting. 

2. Attend a meeting in Houston, TX on December 2 - 3, 2015. 

A. Discuss the 2015 SHFE SRP Statement of Task and address questions about the SRP 
process. 

B. Receive presentations from the HRP Chief Scientist (or his designee), the Space Human 
Factors and Habitability (SHFH) Element, and participate in a question and answer 
session, and briefing. 

3. Prepare a draft final report (approximately one month after the meeting) that contains a 
detailed evaluation of the current IRP specifically addressing items #1, #2, and #3 of the SRP 
charge. The draft final report will be sent to the HRP Chief Scientist and he will forward it to 
the appropriate Element for their review. The SHFH Element and the HRP Chief Scientist 
will review the draft final report and identify any misunderstandings or errors of fact and 
then provide official feedback to the SRP within two weeks of receipt of the draft report. If 
any misunderstandings or errors of fact are identified, the SRP will be requested to address 
them and finalize the 2015 SRP Final Report as quickly as possible. The 2015 SRP Final 
Report will be submitted to the HRP Chief Scientist and copies will be provided to the SHFH 
Element that sponsors the SHFE discipline and also made available to the other HRP 
Elements. The 2015 SRP Final Report will be made available on the HRR website 
(http://humanresearchroadmap.nasa.gov/) . 
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X. 2015 Space Human Factors Engineering SRP Status Review: Statement 
of Task for the Risk of Incompatible Vehicle/Habitat Design and the 
Risk of Performance Errors Due to Training Deficiencies 


The 2015 Space Human Factors Engineering (SHFE) Standing Review Panel (SRP) will 
participate in a Status Review that will occur via a site visit with the Human Research Program 
(HRP) Chief Scientist (or designee) and members of the Space Human Factors and Habitability 
(SHFH) Element. The purpose of this review is for the SRP to: 

1 . Receive an update by the HRP Chief Scientist (or designee) on the status of NASA’s 
current and future exploration plans and the impact these will have on the HRP. 

2. Receive an update on any changes within the HRP since the 2014 SRP meeting. 

3. Receive an update by the Element or Project Scientist s) since the 2014 SRP meeting. 

4. Participate in a discussion with the HRP Chief Scientist (or designee) and the Element 
regarding possible topics to be addressed at the next SRP meeting 

The 2015 SHFE SRP will produce a report/comments from this status review within 30 days of 
the 2015 update. These comments will be submitted to the HRP Chief Scientist and copies will 
be provided to the SHFH Element and also made available to the other HRP Elements. The 2015 
SRP Final Report will be made available on the Human Research Roadmap public website 
(http://humanresearchroadmap.nasa.gov/) . 
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